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ABSx'RACT 

This report presents outcoae data on five referrals 
for chxld-faaily intervention in which a new systea for obtaining 
audio- recorded behavioral data was employed as one of several 
evaluation criteria. The referred child was asked to wear a radio 
transaitter during most of his waking hours at hoae during 
pre-intervention and termination assessments. The transaitter 
broadcast to a receiver- recording apparatus in the home which could 
be activated by an interval timer at predetermined "random" times or 
by parants predetermined "picked" times when the children were said 
to typically exhibit problematic behavior. Daring the project, 
treatment involved an educational program with the parents in which 
they were trained to keep data on their child's behavior and to 
institute behavioral programs to change those behaviors. Results are 
discussed m terms of the frequency of occurrence of child deviant, 
parent negative, and parent commanding behaviors, in general, the 
behavioral outcome data verified that from the two other sour^jes 
Cparent attitude reports and parent collected observation data) , It 
was concluded that the audiorecording procedures provide a viable 
alternative to employing an observer in the home. (Author/SDH) 
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fsj Abstract 

QN nils report presents • outcoae data an five consecutive referrals for 

O chlld-faoiay inwcrvention in whinh a new system for obtaining audio recorded 

^ , ; fedxavloral data was ^enqjloyed as one of several evaluation criteria. The 

referred child was asked to wear a radio transmitter during most of his 
waiang hours at home during pre-intervention-and termination assessments. 
OSie transmitter broadcast to a receiver-recording apparatus in the hcane 
which could be activated by an interval timer at predetermined "random" 
times or by parents at predetermined "picked" times when the children were 
said to typically exhibit problematic behavior. As predicted, child 
deviant, parent negative and parent coaaaaading behaviors were significantly 
higher at the pretest picked times than at the random times. The reductions 
at post-test in all three behavior classes were substantial and significant 
at the picked but not at the random times. Examination of the individual 
subject data indicated, however, that substantial reductions in at least 
two of the three dependent variables occurred in three of the five cases at 
random times. In general, the behavioral outcome data was veridical with 
"that from the two other sources—parent attitude reports end parent collected 
observation data. It was concluded that these procedures provide a viable 
alternative to employing an observer in the home while introducing considerable 
methodological advantages and enabling naturalistic data collection with 
greater convenience and at less cost. 
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EVALUATION OP FAMILY mTERVEUTIOU THHOUGH UNOBTRUSIVE AUDIO RECORDINGS: 

EXPERIENCES IN BUGGING CHILDREN^ 
Stephen M. Johnawi, Andrew Christensen and G. Toa Bellaaor 

University of Oregon 

Bdi87lori8ts working in applied settings have typically relied on 
direct ohservation in the natijral environoent for the design, modification 
and evaluation of treatment programs. In general, they have eschewed the more 
controlled and convenient vehicle of laboratory analogs for clinical 
outcome research, primarily on the grounds that bdjavior in such artificial 
settings may not be representative of behavior in the subjects' natural 
habitat (e.g., see Pawl, 1963; KortlandJ^ 1962; Lobits & Johnson, 197^; 
Martin, Johnson, Johansson, & Wahl, 197^; Milgram, 1963). Yet, a rapidly 
expanding body of methodological literature demonstrates that the representa- 
tiveness of naturalistic observation data, as typically collected, may also 
be affected by a variety of confounding influences. As a resxilt, several 
investigators have pioneered the development of new procedures for direct 
observation, including the use of audio recording (Bemal, Gibson, Williams, 
& Pesscs, 1971; Johnson & Bolstad, 197^) and radio transmitters {e.g., 
Purcell & Brady, 1965; Soskin & John, 1963). 

The present work involved further development and adaptation of these 
methods for assessment in the natural home setting. The a^thod enployed 
represents an integratiaa of the use of radio transmitters as first 
described by Soskin and John (1963) with the use of interval timers to 
8aa5>le behavior at random intervals without subject awareness (as in 
Bemai, Gibson, Williams, & Peases, 1971). In the present report, the proce- 
dure was employed as one of several methods to evaluate the outcome of a 
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b«b«vior modification parent training program though its potential applica- 
tions aro viewed as extensive. 

All of the children involved in the present intervention program were 
asked to wear a small, wireless transmitter during most of their waking 
hours at home in the pre- and post-test assessment periods. The transmitter 
broadcast to a receiver and tape recorder located inconspicuously in the 
home. The recorder was activated at predetermined intervals by a timer 
80 that neither the child nor his parents knew precisely when their inter-> 
actions were being recorded ("random times"). In addition, the recorder 
eould be activated by the parents at preselected times which they had 
identified as most problematic ("picked times''). 

Aa previously indicated, this method was developed in the belief that 
It could at least attentiate many of the data representativeness problems 
documented in the literature. Uhile a c(»aprehensive review of this litera- 
ture cannot be given here, a brief cataloging of these problems and the 
corresponding advantages of this new method of data collection is offered* 
Of particular importance are problems of subject reactivity, observer bias, 
lack of generalizability of observer agreement estimates, and lack of 
representativeness in time sampling procedures. For comprehensive discus- 
sions of these problems, the reader is referred to several current revievs 
on the topic (e.g., Johnson & Bolstad, 1973; Jones, Reld, & Patterson, 
1974; Lipinski & Nelson, 1974). 

Stab j act reactivity concerns the possibility that being observed can 
affect one*s behavior. A potential problem in all naturalistic observation, 
subject reactivity appears to be especially important in the home environ- 
ment, where a live observer can be a particularly intrusive addition to a 
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typically private and constricted setting. Several aspects of the observa- 
tion process may affect the observed behavior and lead to unrepresentative 
data. First, it is possible that expectations held by observers may 
Influence the subjects' behavior. Observer expectancies may be communi- 
cated in some way to subjects who then alter their behavior while being 
observed. Evidence that this expectancy effect can obtain has been pro- 
vided by Bosenthal and Fode (1963) , Rosenthal and Jacobson (1966) , and 
Foda (1965). 

Jloro generalized changes in subject behavior can ai «- c^ccur during 
observation. It has been demonstrated, for example, that ■. Ejects may 
avoid situations in which they can be observed (Bechtel, Vi67) or may 
decrease overall activity level during observation (White, 1972). Of 
course, if these generalized changes in behavior remain stable, this sort of 
reactivity, although providing a biased estimate of unobserved behavior, 
would not necessarily preclude accurate intrasubject measures of treatment 
effects. Considerably more threatening to the representativeness of 
bahavioral data, however, is the posailility that specific response sets 
might result from demand characteristics in the observed settings. During 
borne Observations prior to family intervention, for example, parents might 
wish their child to appear deviant in order to insure acceptance for treat- 
ment. Similarly, parents might wish to present their child as more normal 
after treatment in order to express appreciation to therapifts and justify 
termination. That parents can, in fact, manipulate childrens' behavior in 
response to instructions to do so has been demonstrated in two studies 
(Johnson & Lobitz, 1974; Lobitz*^^ & Johnson, 1974). 
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Th«ae investigations denKmstrated the troublesome possibility that the 
effects of observation may interact with demand characteristics accompanying 
treatment assessment to alter subject behavior « That such Interactions can» 
in fact» occur has bef:n demonstrated In a classroom setting by Kenty Fisher 
and O^Leary (1974). these investigators found that, relative to periods of 
inconspicuous observation, children's off*- task behavior increased when an 
observer was present In the classroom prior to treatment, but decreased as 
« result of observer presence during treatment* Thus, while covert assess-* 
meat of of f- task bahavlor revealed an Increase in off"»task behavior during 
treatment, measures obtained by an observer in the ciassroom Indicatad a 
decrease. The podsiblllty of such differential reactivity during different 
observational periods is present in much behavioral research* 

Several characteristics of the audio recording system used In the 
present work appear to diminish subject reactivity problems* First, the 
procedure is clearly less Intrusive than sending a live observer into the 
home and avoidi many of the restrictions on bdhiavior of family members 
necessary with direct observation (e*g*, restrictions in movement in the 
home). Second, the possibility that observers* expectancies may affect 
e«^Jects* behaviors is avoided entirely* Finally, the mechanical operation 
of the recording device, which prevents exact knowledge of when data is 
being collected, should minimize the possibility of generalized changes in 
behavior or the operation of more confouQding, specific response sets* As 
a result of these factors, data obtained by the audiotape procedure shoixld 
be more representative of the subjects' unobserved behavior* 

Concern for observer bias In behavioral data reflects the possibility 
that systanatic recording errors may occur as a result of the observers^ 
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kaowXedge of the hypotheses under 8c idy, or the status of observed subjects. 
Observer bias represents a threat to data representativeness because of 
observers* errors, rather than subjects' behavior. That such Information 
can, in fact, result in systematic recording errors is provided by Azrln, 
Hola, Ulrich, and Goldiaajond (1961), Rapp (1966), and Kass and O'Leary 
(1970). In the present context, knowledge of whether a family observation 
occurs before or after treatment represents a potential source of observer 
bias. Since this information is likely to come from the family itself 
during live home observations, the audio recording procedure used in the 
pr«8ent work again seems preferable. There is no necessary correspondence 
between the order of coding tapes and the actual order of this recording 
occurrmce, and observers can be reassigned as soon as biasing information 
occurs in a recorded Interaction. ^ 

The generalizability of observer agreement data is at issue whenever 
observations are intermittently monitored for agreement. Observer agree^ 
nent statistics are typically obtained in natural settings by introducing a 
second, Independent observer into the situation and comparing the two 
records. There is considerable evidence that the resulting measures over- 
estimate agreement at other times (Reid, 1970; Taplln & Reid, 1973; 
Demaster & Reid, 1972; Romanczyk, Kent, Diament, & O'Leary, 1973). 
Observer agreement statistics computed when observers were aware of being 
checked were considerably higher in these studies than when observer 
agreement was monitored covertly. In most natural settings, however, covert 
assessment of agreement among live observers is practically Impossible. 
Thus, the present audio recording system again seems preferable, inasmuch 
as the coding of audio records can be covertly checked for agreement without 



Johnson, Chrlatensen* and Bellesi^ 



BEST COPY PSmm^ 



6 



difficulty. 

A final concern with traditional methods of naturalistic observation 
is the difficulty and expense involved in obtaining an adequate time soople 
of Important behaviors. Behavioral measures vhlch are obtained In limited 
time periods may be either unrepresentatif-'e of overall behavior or insensi- 
tive to changes at other times. For .e, an adaptation of the behavioral 
coding system developed by Patterson, Ray, Shaw, and Cobb (19^9) has been used in 
this laboratory to record Interactions am>ng family members immediately 
prior to the evening meal. Although same significant changes in behavior 
vecorded at these times did occur as a result of treatment, the overall rate 
of deviant behavior was not affected and the results on these observational 
BM^ures have not paralleled the magnitude of change recorded by a variety 
of other outcome measures at either termination (Eyberg & Johnson, 197^) 
or follow-up (Johnson & Christensen, 1974). The times sampled by this 
ho^ observation systemi simply may not have been representative enough to 
reveal actual changes. Altiiough Patterson (1974a, b) has shown significant 
reductions In deviant behavior after treatment using this method, Perber, 
Keeley and Sba&berg (1974) have also been unable to do so and have discussed 
this failure in light of the representativeness problan. 

Zt was expected that the audiotape procedure would provide both a more 
sensitive and comprehensive outcome measure than direct observation in the 
home. First, a much larger part of the day could be sampled. The recorder 
was mechanically activated for a fifteen minute period at breakfast time, 
before dinner and after dinner each day without the f«aily's awareness 
("r^mdom times"). Second, provision was made for the parents to activate 
the recording devices for fifteen minutes daily at a time which they had 
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IdAACified as the most problematic ("picked times"). 

It WIS hTppthesUed that (a^ prior to treatment, the rate of child 
deviant, parent negative and parent coasaandlng behaviors would be greater 
during the picked times than during the random times; (h) the rate of the 
three behavior classes would decrease from pretest to post-test during the 
picked times; and (c) there would be greater decreases in these behaviors 
during the picked times than during random times. Reductions during the 
random times were hoped for but not predicted on the basis of prior failures 
to show changes in overall suBaaaries of child deviance as a function of 
treatment (Eyberg & Johnson, 197U; Johnson & Christeasen*, 197^*). 

Method 

Subjects ' 

Five families with children believed to e^lblt active behavior problems 
la the home participated in the treatment program. One family contained two 
children who were treated in the p-rogram. "Active behavior problems" was 
used to refer to aggressiveness, destructlveness, disobedience, hyperactivity, 
temper tantrums, or high rate activity with annoyance value. Families were 
not accepted into this program if the problem child was severely retarded, 
had experienced severe and documented brain damage or exhibited behavior 
problems vrtilch would ordinarily cause him to be labeled "autistic" or 
"shlzophrenlc." 

Pour families had both parents in the home while one was father-absent. 
The educational level of parents ranged from 8 to 14 years with the mean of 
11.5. Income level for famlUes ranged form $1,800 to $10,000 annually 
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with a mean of j^proximacely $6,000. The treated children ranged In age 
im 4 to 10 with a mean age of 6.6. 
Treatment Procedures 

Six advauced graduate student therapists and one supervising therapist 
participated In the treatment program*^. Therapists were formed Into 
permanent tvo^person teams with both team members typically participating 
in each treatment session. 

The treatment approach was educational In nature and based on the 
principles of social learning theory and the techniques of behavior modifi- 
cation* Uany of the procedures were patterned after those described by 
Patterson and his associates (e.g., P&tterson, Cobb, & Bay, 1973; Patter- 
eon, Bay, Shaw, iS68; Patterson & Keld, 1973). Through a series of no more 
than twelve weekly sessions, parents were trained to keep data on their 
child's b£<'aa\'ior and to institute behavioral programs to change those behav~ 
iora. The treatment procedures have been described In greater detail by 
Eyberg and Johnson (1974). 
Measures of Treatment Outcome 

There were four measures of treatment outcome: parent observation 
data* two parent verbal report measures, and audio-tape observational data 
taken in the home. 

Parent observation data . The therapists required both i»arent« to 
record the to-be-treated child problem behaviors for a one-week baseline 
period prior to beginning an Intervention for them. Parents recorded the 
frequency and/or duration of the behavior of interest for a specified time 
each day. Recording time varied from one-half-hour per day for very high 
rate or situation specific behaviors to the entire day for lower-rate 
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bth«nors. During intervention, behavioral recording continued as long as 

■ t> 

tiie behavior remained a focus of treatment. 

Verbal report measures . A form of the Becker (1960) Bi-Polar Adjective 
Oiecjaist was employed to obtain parental description of the treated child.* 
This form has previously been employed by Patterson, Cobb, and Ray (1973) 
to assess parents' change in perception of their children following treat- 
BMmt. 

Xb& Therapy Attitude Inventory was constructed specifically for 
•arller research on the assessment of parents* satisfaction with the process 
and outcome of this treatment program and with the therapists involved 
(Byberg & Johnson, 1974). 

Home observations. The audio equipment used for home observations 
consisted of a transmitter (Edcor PM-1), a receiver (Edcor St-3 sensatuner), 
« 24-hour timer (Intermatic Model T 1975), two 15-mlnute time switches 
(Mark timers), and two reel-to-reel tape recorders. This particular radio 
package was selected in order to minimize the possibility of accidental 
pickup of recorded behavior by others. This system employs a relatively 
Uttle used, low range band typically employed by businesses to coamunicate 
with their representatives in the field. Dial scanning by these special 
users would presumably be minimal. And, the transmitter's maximum range is 
approximately 200 feet. All of these factors would make accidental pickup 
by other users extremely unlikely, but all families were informed of this 
risk. 

The transmitter (dimensions: l-l/4"x 2-3/4" at 3-7/8"; weight, 11 02.) 
was worn on the belt of the target child for time blocks when he was inside 
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th« hoBtt. Tine blocks included morning, afternoon, and evening periods for 
« cotal of approximately 5 hours a day. The traii{imitt«.r broadcast continual- 
ly during these periods. 

All other eiiuipaent was stored inside a trunk placed in some convenient 
location in the home (e.g., a closet). The receiver was wired to two tape 
recorders, one of which was activated at 3 predetermined IS-oinute ''random 
tiaea" each weekday. These randoa tiates were usually prograsraed to occur 
Once around breakfast, once in the late afternoon and once in the evening. 
Tba target <^ild and parents were aware that some segmcmts of their inner- 
action would be recorded, but they did not know of the duration or exact time 
of the recordings. The tiiser was prograosaed so that it would not activate 
tba random tape on weekends. 

The second tape recorder could be activated by the parents operating 
another 15 -minute timer located on the outside of the trunk. Thus, there 
was a potential for three 15 -minute segments of rand<Mly selected audio 
material and one 15 -minute segment of picked material pttr day. The p«xents 
were instrticted to operate this "picked tape" once a day during a regularly 
problematic time or situation. Four famlll€u; selected bedtiism as the most 
pn»blematlc situation and were Instructed to turn the tape recorder on at 
€ha point in the bedtis;e sequence at which behavior problems usually 
occurred. It was emphasized that the same setting evcmt should be used on 
•ach occasion. One set of parents was unable to specify a problematic 
situation , and they were instructed to turn the tape recorder on at a 
spedflad time in the afternoon whaa the child was said to be particularly 
problematic. 
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The therapists in each case brought the taping apparatl to the home for 
both aasessments and introduced the family to its operation. The parents 
were provided with written as well as oral instructions in the use of this 
i^ulpment. The Instructions emphasized that parents were not to inform 
their children about the taping at picked times. 

In all cases but one, the recording apparatus was kept in the home for 
five to seven consecutive week days. One family failed to follow the picked 
time Instructions properly during part of the first week of baseline and 
consequently was required to repeat the aasesftaent for a second week. All 
rsadom time data was used for this family. Due to fluctuations In faxally 
8Chedul«ss or failures in following instructions, less than the maximum 
amount of data was collected. The mean number of 15-minute random time 
segments at each testing period was 9.10 (2 hours 16.5 minutes) with a range 
of 6-11 for all regular assessments and a high of 16 segments for the one 
special case at baseline. The mean number of picked time segments was 5.20 
(1 hour 18 minutes) per assessment with a range of 4 to 7. 

Three means of censorship were provided to protect the families* privacy: 
(a) The family could activate a censor switch located on the outside of the 
trunk, disconnecting the receiver for 15 minutes, (h) The family could 
listen to and erase any part of the tapes prior to their being listened to 
and coded by observers, (c) The family could listen to and erase any part 
of the tapes after their coding by observers. Confidentiality of all 
aasassment materials was assured and censorship was not discouraged. None 
of the families reported using the censor switch and none requested that 
they listen to or erase any tapes. 
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Obsarvfttlonal system , A new observational code, dertved In part froa 
the direct observational system devised by Patterson, Ray, Shaw and Cobb 
<3S69), was developed for use with the audio tapes. A slightly shortened 
form of the present system was used in a recent study by Johnson and Bolstad 
(197 10* The coding system was designed to provide obs ^ "Nation data on all 
of the verbal behavior of the Individual wearing the transmitter and all 
varbal behavior of those who interact with him. 

The tapes were coded by two observers trainetl comprehensively prior 
to coding these tapes who practiced weekly to maintain coding ptoficiancy. 
The first observer coded all tapes and was frequently aware of codisig tapes 
representing baseline or termination assessment. Thirty-six of the 1A2, 
15 -minute tape segments (or 25%) were randomly selected and coded by a 
second observer. The second, calibrating observer was unaware of the base- 
line -termination status of the tapes being coded. Thus, calibrations done 
on these tapes are representative of the r^naining noncalibrated data. 
To the extent that observers agree on the calibration tapes, an absence of 
observer bias is indicated. 

Observers coded the tapes in 10 -second blocks demarcated by clicks 
recorded on the second channel of each interaction tape. The coding system 
eontainjB l6 discrete codes which were reduced to four summary statistics for 
the purposes of this research. These summary codes were child deviant, 
parent negative, parent commanding and sibling negative behavior. Because 
this system is relatively new, the computation of dbserver agreement was 
rather comprehensive. Observer agreement percent was computed on each coda 
and each summary statistic. To count as an agreement, both observers were 
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required to see tlie same event in the saae lO-aecond Uock. Agreement on 
nonoccurrence was, of course, not counted. In addition, correlations of 
the rate per minute scores obtained by each observer in the 36, 15-minute 
blocks were ccsnputed. Those codes which were used less than five timen were 
not subjected to either analysis. A Hsting of the codes and summary statistics, 
together with their associated calibrated occurrences and agreement figures is 
prcsented in Table 1. The code coii?)osition of each suimaary statistic is also 
Indicated ii this table. 

Insert Table 1 About Here 

The mean percent agreement for individual code categories was 82% with a 
range from 70?^ to 91%. The mean percent agreement for summary statistics was 
02% with a range from 77% to 86^. The mean correlation for individual code 
categories was .9U with all codes but one showing correlations of .92 or better. 
Smmjary statistics yielded very high agreement correlations of .97 or hi^er. 
Foar the data analysis involved in this study, the agreement figures on summary 
statistics are most directly relevant. 

Results 

Verbal Report Measures 

All nine parents involved in this treatment program gave more favorable 
descriptions of their children af termination than at baseline on the basis 
of the Becker (i960) Bi-Polar Adjective Checklist. This checklist was 
scored on the basis of five factors derived by Patterson (as in Patterson, 
Cotob, & Ray, 1^73). UnpubUshed data from the present laboratory shows 
that three of these factors (Tense Disposition, Aggression, and Conduct 
Problems) are highly correlated, and the sum of these three factors has 
proven to be a useful and valid single index from this measure (Lobitz, G. 
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& Johnson, 1974; Johnson & Chrlstensen, 1974). The baseline and tenalna- 
ti(m scores of all nine parents were compared using a paired observation 
jt~test. In the family with two treated children, only the parents' ratings 
of the child initially rated as most deviant were included for analysis* The 
Analysis showed that the reliable Ijnprovement was statistically siguific^t 
(t • 4.83, i£ - 8, £ <*005)s 

The Therapy Attitude Inventory included seven items concerning the 
parents* ratings of trcratment outcome, and three iteo» reflecting parent 
ratings of the therapists* All items were rated on a scale from 1 (indicat- 
ing maximum dissatisfaction or deterioration in condition) to 3 (indicating 
maximum satisfaction or improvement) * Parent ratings were averaged in the 
family with two treated children. For all parents combined, the average 
rating of those items relating to therapy was ^-37, or between somewhat favor-* 
able and very favorable. The mean score for all parents on the questions 
csonceming ratings of the therapists was 4.33. Thus, both verbal report 
TOasures indicated favorable outccH&e for all cases in agreement with previous 
research involving this treatment program and these verbal report measures 
(Syberg & Johnson, 1974; Johnson & Chrlstensen, 1974). 
Parent Observation Data 

Parents collected behavioral data on each child b^avlor problem treated. 
The average number of problems per family was 6.8 with a range from 4 to 10. 
The outcoue restilts based on parent data were computed on the basis of the 
percent reduction from baseline observed in the last three weeks of active 
treatment for the individual behavioral problem. An average percent reduct- 
ion in child deviant behavior was then computed for each family. The 
average percent reduction per family was 77Z with a range of 58% to 91%. 



BfST copy fiu-. ... 

Jdmson, Christensen, and Bellaay 15 
This estimate is also quite consistent with previous research using the saxoe 
treatnent program and observational methods. 
Audio Tape Data 

The audio tape data for each case on each of the three dependent variables— 
child deviance, parent negativeness , and parent commands—are presented in 
Table 2.^ 

Insert Tables 2 and 3 About Here 

Four repeated meaaixres, planned coii5>arisons , were performed on each dependent 
variable reflecting: (a) a comparison of b^avior rates at the pretest 
between picked and random times, (b) a comparison of the pretest and post-test 
behavior rates at the picked time, ic) a comparison of the pretest and post-test 
behavior rates at random times , and ( d) a test of the interaction between taped 
time and assessment time. For each variable, significant effects were predicted 
for comparisons a, b , and d but not for comparison £, The results of these 
aaaOyses are presented in Table 3. 

As predicted, the rates of child deviant behavior were much higher at 
baseline in the picked times than in the random times. Child deviant behaviors 
occurred at l.TO per minute in the baseline picked itmes, but onOy at .50 per 
minute at the baseline random times (t - 5.01, df = 8, £ < .005). The direction 
of this difference was the same in every case. Furthermore, the deviant behavior 
rate per minute was reduced at termination in every case to an average of ,50 
per minute (t « 5.98, ^ « 8, 2. < .005)' On the average, the child deviant 
behavior rates did not change during the random times and the interaction 
between taped time and assessment time illustr'ited in Figure 1 was significant 
(t = k.k9, df « 4, £ < .01). 
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Insert Figure I about here 

Sxanlxiatio& of the Individual subject daf.a In Table 2 Indicates that 
there was a substantial drop in child deviant behavior during the random 
time for two cases, essentially no change for two others, and a rather 
sitbstantial increase in the fifth case. 

The same pattern of r.^sults is repeated for parent negativeness. 
Parent negative behaviors were significantly more frequent at the picked 
than random times at baseline. The parent negativeness decreased reliably 
and significantly during the picked times, but not during the random times, 
■ad the interaction was significant. Examination of the individual subject 
data in Table 2 reveals substantial reductions in parent negativism in two 
casas, little change in two others, and a very substantial increase in case 
number five. The reductions in parent negativism in case numbers two and 
four in both the picked and random time segments are noteworthy in that 
reduction in such behavior was an explicit target of the family intervention. 
Parents in both cases were taught to use time-out procedures and instructed 
to eliminate all other forms of aversive control such as spanking, yelling, 
threatening, etc. 

Tba pattern of results was similar for the parent command variable. 
Parent commands were significantly higher on the average during the baseline 
picked times than the baseline random times. Parental commands declined 
reliably and significantly during the picked, but not during the random 
times, and the interaction was significant. Examination of the individual 
subject data in Table 2, however, reveals noteworthy reductions in the 
parent command rate during the random time for four out of the five cases, 



BEST copy mmi^ 

Johnson, Christensen, and Bellaioy ^,7 

but • «ub«taatlal lncre«ae in coosaand rate In case number five. The aubitan- 
eial reductions in cases two and four are particularly noteworthy since 
parental command rate was a discrete target of treatment In these cases. 

In summary, all three predictions were confirmed on all three variables. 
Child deviant, parent neg&tive and parent commanding behaviors were signifi- 
cantly higher at baseline during the picked than random times. All behavior 
categories demonstrated significant reductions during the picked times but 
not during the random times and all interactions were significant. The sib- 
lin« negative variable was not analyzed because siblings were coded in oxily 
two cases and were quite inactive when present (see Table 1). 

Although the random tape data did not show the consistent declljies In 
child deviant, parent negative, and parent commanding behaviors demonstrated 
at the picked times, examination of Table 2 Indicates that fairly substantial 
reductions In at least two of these variables was observed for three of the 
five families (families two, three and four). In the authors' view, the 
random tape data provide additional evidence for the effectiveness of the 
treatment program in these three cases. There was a noticeable reduction 
in the parent command rate In case number one, but all other changes at 
random times were negligible. The random tape results for case number five 
are rather discouraging in showing an increase in every dependent variable 
at the post-test random times. If reliable, this data may reflect the 
operation of contrast effects as documented in some other behavior modifi- 
cation research (e.g., see Johnson, Bolstad & Lobitz, 1974; Meichenbaum, 
Bowers & Ross, 1968). In any case, all of these families will continue to 
have follow-up assessment on all measures and additlona: treatment will be 
aade available as necessary. 
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Discussion 

Experience with these radio**assisted audio recording procedures has 
ooaviaced the preseut investigators of their considerable utility for 
assessment in the home« The procedure's appear to solve xaany of the repre** 
sentatlvaness problems as outlined earlier. On logical grounds the 
relatively unobtrusive transmitter siiould yield less subject reactivity and 
the constant presence of the device during most waking hours should promote 
more rapid adaptation. In addition to lessening generalised reactivity to 
being observed t these advantages would seem to eliminate observer communl'^ 
cation of expectancies to the observed and lessen the possibility that axxh^ 
jects' responses would reflect only the demand characteristics of the assess 
ment situation. 

In many applications » these procedures would enable investigators to 
keep observers totally uninformed of the hypotheses tested or the circum- 
stances of assessment^ andf In cases where this Is impossible > the procediires 
for checking observer agreement can solve the bias problem If observer relia 
blllty remains hlgh# Observer agreement checks can be made randomly^ as In 
the present case, by a second » uninformed observer yielding completely 
generallzable reliability assessments « In this study ^ there was nearly 
perfect agreement in observer fluctuations on the three basic summary 
statistics (correlations - *97-*99)« Additionally^ and perhaps most 
Important t these procedures can be used to sample behavior at any time. 
The data presented here would indicate that this provision Is particularly 
useful for treatment evaluation. Although the procedures for defl&ing the 
"picked" times employed in this study did not eliminate subject awareness » 
high density "random" taping at specific intervals would presumably have 
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thl« effect. 

The present experience suggests that audio taping yields behavioral 
data of sufficient density at even random times to be meaningful. Further- 
more, observer agreement in coding this data was high by every criteria 
employed. And. the present experience suggests that naturalistic beiiavioral 
data can be collected with these procedures at considerably more convenience 
to all involved and at less expense than is typically encountered in employ- 
ing observers in the home* 

The tape data results at picked times were highly veridical with 
parent data and parent attitude reports. The picked tape data showed a mean 
reduction in deviant behavior of 70% which is comparable to the average 772 
reduction in targeted behavior problems as recorded by parents. In the four 
cases where bedtime problems were targets of intervention, parent data showed 
an average reduction of 79% in such problems. Although not statistically 
significant, the random tape results showed substantial reductions in two 
or aore of the dependent variables for three of the five families. The 
verldicality of this outcome data is particularly encouraging in view of 
previous research in which behavioral data has failed to substantiate the 
degree of treatment effectiveness demonstrated by other data sources 
(Eyberg & Johnson. 1974; John^ion & Christensen, 1974). The present 
results were completely consistent with the trends observed in this earlier 
research, however, in that behavioral data showed greater improvement as 
it was more specifically related to the behavior problems treated. Thus, 
In this earlier research, parent data on targeted problems showed the 
greatest improvement, followed by a significant but modest index of 
improvement based on observer records of those codes related to treated 
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problems fpllowed by nonslgnlf leant change on a general index of observed 
dtvlance (Eyberg & Johnson, 1974; Johnson & Chrlstensen, 1974). 

The specificity of the taping system Is probably responsible for the 
•ahanced agreement among data sources obtained In this study. Since these 
behavior modification treatment procedures aia frequently aimed at very 
situation and time specific behavior problems, the assessment procedures 
oust possess sufficiently narrow •'bandwidth" to yield high "fidelity." 
Thus, with only 5 subject families, child deviant, parent negative and 
parent commanding behaviors were significantly higher at the baseline 
picked times than at the baseline random times. All three variables 
dacreased significantly from baseline to termination at the picked but 
not at the random times. In every case, the reductions at the picked 
times were significantly greater than those at the random times. 

It must be acknowledged that the picked and random tapes also differed 
on the dimension of parent awareness of recording. The change in behavior 

££Hii be a functlor of this awareness and the associated response 
of parents to demand characteristics. The available data would suggest 
that parents of generally "deviant" children can behave so as to cause 
their children to act significantly more deviant than usual but not aignlfl- 
cantly less deviant (Lobltz, W. & Johnson, 1974). On the average, 
children in the present study behaved much more devlantly at pretest 
picked times than they did at the "usual" random times. At the post-test, 
the mean picked deviant behavior rate was virtually identical to the mean 
rate at random times. Thiis, the data could merely reflect a "fake bad" 
set at baseline. Several important considerations mitigate against this 



BEST COPY AVAILABLE 

Johnson, Chris tens en, and Bellaoy 21 

lat«rpr«t«tion. The first comes from consideration of the Walter and Gil- 
«ore (1973) study in which a placebo intervention program was employed for 
families with child problems quite similar to those dealt with here. Although 
parental expectancies were successfully manipulated by this placebo proced- 
ure, observers who employed the presumably more obtrusive method of going 
Into the homes, recorded slight increases in child deviant behavior at the 
post-test. Furthermore, parental ratings of child sympton occurrence showed 
ao change with placebo treatment. The second consideration derives from a 
atudy (Lobltz, G. & Johnson, 1974) in which the pLreteet deviant behavior 
rates of referred and normal children were compared with similar behavior 
rates of comparable groups in which parents had been asked to "fake good," 
behave normally or "fake bad." The comparisons showed that the deviant 
behavior means of both referred and normal children corresponded most 
closely to the "fake good" condition means of the appropriate comparison 
groups. These results suggest that most families in a home observation 
situation may respond more to a generalized social desirability response 
set rather than to any possible demand characteristics associated with 
treatment, other results which argue against the demand characteristic 
hypothesis in the present research include the fact that the parent data 
was veridical with the tape data in showing high levels of deviant behavior 
during the picked times prior to treatment and subsequent decreases after 
treatment. In addition, substantial reductions in child deviant behavior 
also occurred at random times for two cases and substantial beneficial 
changes in parent variables occurred for three cases. 

In spite of all these considerations, the logical possibility of the 
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4«mattd charactarlstlc hypothesis still exists. As Indicated earlier » the 
•yetem could be adapted for high density random taping at Intervals likely 
to be problematic. Although less efficient than the present procedures, 
this change would appear to solve any problems Introduced by differential 
afwareness and would, as an additional advantage, eliminate the need for 
parental manipulation of the equipment. 

It must be recognized that the present study is purely descriptive in 
that no control group was employed. Three control group studies have been 
completed, however, on very similar populations with comparable treatment 
and assessment procedures including parent report and observational data. 
TLa Walter and Gllmore (1973) study involved comparison with an attention 
placebo group, while the other two studies involved comparlaons with no 
treatment groups (Johnson, Bolstad, & Lobltz, 1974; Wlltz, 1969). The 
results of all studies have been consistent in showing either no change 
(Johnson, Bolstad, & Lobltz, 197A) or slight increases in observed 
deviant behavior in the control or placebo groups (Walter & Gllmore, 1973; 
Viltz, 1969). An absence of significant change in parent ratings of the 
target child was obtained by Johnson, Bolstad, and Lobltz (1974) and Walter 
end Gilmore (1973) found no change in parent ratings of the child's symptom 
oeeurrdice with placebo treatment. Thus, although not completely comparable, 
the available research would tend to indicate chat the kinds of changes 
observed here would not be observed in similar groups exposed to placebo or 
no->treatment. 
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the authors on request. 

^Case number 3 involved a family in which a 9- and 10-year-old boy 
were both involved in the treatment program, and each boy wore the transmitter 
on alternate days. This was the only family in which siblings could not be 
reU^ly discriminated in coding. As a result, all child and parent behaviors 
w«re coded only in those segments when both children were together and all 
data was averaged. These procedures resulted in the loss of only 1.33 minutes 
of data in pretest picked time and 12 minutes in the post-teat picked time. 
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Table 1 

Observer Agreement Con^iuted by Percent and Correlation 
For Sach Code and Summary Statistic 







Total 










Call orated 


Percent 






Code 


Occurrence 


Agreement 


Correlation 








93 


.99 




rositive Lommana 


193 


ol 


.98 




Negative Command 


39 


79 


.59 


• 


Suggestive Command 


IT 


76 


.95 




nepeated ^/Onmaxid 


131 


07 


.99 


# 




0 






## 


uls approvai. 


3 




1 


f •# 


Threat 


30 


73 


.92 


*## 


Whine 


179 


8U 


.96 


!•# 


Yell 


219 


80 


; .97 


t«* 


Verbal Negativism 


82 


70 


.96 


t»» 


Tease 










Physical Negativeneas 


\ 






«fH» 


Disapproving Tone 


l6k 


73 


' .93 


f 


Cry 


101 


97 


.99 


f 


Demand .ittention 


82 


78 


! .99 




Commands 


379 


86 


i .99 




Parental Negativeness 


i 239 


77 


.97 




Sibling Negativeness 


I 6 


83 


1.00 




Child Deviant j 622 


' 83 


.99 



* Codes included in the command category. 

•* Codes included in the parent or sibling negative category. 

* Codes included in the child deviant category. 
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Table 2 

Rate per Minute of Child Deviant , Parent Negative 
and Parent Coaaaanding Behavior at Pretest and Post-Test 
at Random and Picked Times 



T 

i 



'j 



jl_J*reteat 



Picked Times 
Post-Test 



Child Deviant 

Case 
1 
2 
3 
k 

5 

Mean 



Parent Negative 
Case 

1 ; 

2 
3 
k 

5 •; 
Mean 

Parent Commands 



Case 
1 
2 

3 
k 

5 

Mean 



.683 
1.8i23 
1.U80 
2.680 
1.82U 

1.6T0 



% Change 



Random Times 



.1*50 
.253 
1.356 
1.31*0 
.757 

.831 



Pretest 



.197 
.883 
.333 
.686 
.Ul3 

.502 



-715? 
-52^ 
-78J5 
-7U55 
-17% 

-70% 



.835 
.291 
.606 
.3lU 

.500 



Post-Test 



.082 
.250 
.500 
.220 
,kkO 

.298 



-82? 
- 1% 
-63% 
-8U55 
-k2% 

-5h% 



.370 
.UUl 
.080 
.697 
.153 

.31*8 



.515 
.7U 
.073 
.3U5 
.773 

.U83 



.300 : 


.016 


i 

-95% 


.218 ' 


.187. 


.101 ; 


.050 


-50? 


.181 


.070 


1.120 ■ 


.222 


-80? 


.028 


.027 


2.206 ; 


.372 


. -83? 


.598 


.1U2 


.392 ! 


• 321f 


; -U2? 


j .036 


.288 


.82U j 


.177 


! -70? 


.212 


.11*3 



.269 
.187 
.036 

.233 
.1*39 

.233 
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Table 3 

Plaoaed Coaxparison t_-teat8 on Child Deviant, Parent Negative and 



Parent Coiumanding B^avior 



a 





Variable 


Comparison 


Child 
Deviance 


! Parent 
! Negative 


1 Parent 
Command 


Picked vs. Random Times 
at Pretest 


5.01*«#» 


2.63** 


2.U8" 


Pretest vs. Post-Test 
at Picked Times 




2.6l«* 


3.20»»» 


Pretest vs. Post-Test 
at jrtandom Times 


.08 


.30 


.69 


Interaction: Taped Time 
X Assessment Time*^ 


1 


2.28» 

t 


2.21* 



*L < '05 
**£ < .025 

< .01 



All tests are one-tailed (df = 8 for first three comparisons; 
df a for Interaction) , 

*^e interactior for child deviance is illustrated in Figure 1. 
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